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Abstract 

Background: Teacher training may improve teaching effectiveness, but it might also have paradoxical effects. 
Research on expertise development suggests that the integration of new strategies may result in a temporary 
deterioration of performance until higher levels of competence are reached. In this study, the impact of a clinical 
teacher training on teaching effectiveness was assessed in an intensive course in emergency medicine. As primary 
study outcome students' practical skills at the end of their course were chosen. 

Methods: The authors matched 18 clinical teachers according to clinical experience and teaching experience and 
then randomly assigned them to a two-day-teacher training, or no training. After 14 days, both groups taught 
within a 12-hour intensive course in emergency medicine for undergraduate students. The course followed a clearly 
defined curriculum. After the course students were assessed by structured clinical examination (SCE) and MCQ. The 
teaching quality was rated by students using a questionnaire. 

Results: Data for 96 students with trained teachers, and 97 students with untrained teachers were included. Students 
taught by untrained teachers performed better in the SCE domains 'alarm call' (p < 0.01) and 'ventilation' (p = 0.01), 
while the domains 'chest compressions' and 'use of automated defibrillator' did not differ. MCQ scores revealed no 
statistical difference. Overall, teaching quality was rated significantly better by students of untrained teachers (p = 0.05). 

Conclusions: At the end of a structured intensive course in emergency medicine, students of trained clinical teachers 
performed worse in 2 of 4 practical SCE domains compared to students of untrained teachers. In addition, subjective 
evaluations of teaching quality were worse in the group of trained teachers. Difficulties in integrating new strategies in 
their teaching styles might be a possible explanation. 
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Background 

Teacher training may improve quality and professionalism 
of teaching and is therefore widely employed in faculty de- 
velopment programmes [1-9]. However, training may have 
the paradoxical effect of reducing actual performance. Re- 
search on expertise development suggests that the integra- 
tion of new strategies could result in deterioration of 
performance until higher levels of competence are reached. 
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As a result of new learning, current routines, procedures, 
and organization of knowledge, which had worked quite 
well before the training, may be called into question. In 
particular, meta-cognitive components of training may 
make learners aware of their potential for further develop- 
ment [10-12]. This awareness may reveal one's own deficits 
to the trainee, and may therefore cause insecurity [13]. 

Furthermore, developing and integrating new strat- 
egies into a routine requires considerable time, re- 
training and practice [14], often referred to as deliberate 
practice [15]. As a consequence, during an intermediate 
time the training might therefore lead to detrimental ef- 
fects. Evidence exists that such U-shaped developments 
occur while new information or new skills are being 
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worked into an existing teaching routine: When new 
paths are entered, some drawbacks in performance are 
to be expected. In the field of clinical teacher training 
however, this potential phenomenon has not yet been 
clearly addressed. We therefore wanted to explore the 
impact of clinical teacher training at an early time point 
after intervention. To measure quality and effectiveness 
of teaching we chose teachers' performance in an inten- 
sive course in emergency medicine. Using a prospective 
randomized controlled design, the investigators were 
blinded towards teacher training and teachers were 
blinded towards the final purpose of the study. At 
present, the Objective Structured Teaching Exercise 
(OSTE) is the most extensively studied instrument to as- 
sess teaching quality [16]. However, we preferred to use a 
different outcome parameter since the ultimate goal of 
teaching in emergency medicine is to improve students' 
skills and the outcomes for patients. With these goals in 
mind, we chose practical resuscitation skills as the primary 
outcome parameter. We assessed: 

• students' practical and theoretical performance at 
the end of the emergency medicine course, 

• teaching quality rated via evaluation via participating 
students, and 

• self-perception of teachers. 

Methods 

In August 2006 eighteen clinical teachers from emer- 
gency medicine and anaesthesiology were matched with 
respect to their clinical experience and teaching experi- 
ence, and then randomly assigned to a session of teacher 
training lasting two days ('trained group') or no training 
('untrained group'). In anticipation of organizational 
shortcomings within the department, we planned one 
additional pair of teachers as backup. From October 
2006 to June 2007, all of the teachers in the sample were 
routinely scheduled to teach emergency medicine to 3 rd 
year students. [In Germany, medical students start their 
curriculum after 13 years of school. Their first two years 
at university predominantly cover basic science] . 

Characteristics of teachers 

All clinical teachers were employees of a university teach- 
ing hospital. None had received any teacher training prior 
to the study. For each emergency medicine course, a team 
of three teachers was responsible, at three different levels 
of clinical experience. These levels were defined by the 
learning content, which each teacher had to cover in his 
or her specific sessions: 'juniors' had a clinical experience 
of 0-1 years (in clinical anaesthesiology), 'intermediates' 
3-6 years (additional experience in intensive care and 
emergency room service), and 'seniors' 7 or more years 
(additional experience in pre-hospital emergency 



medicine). In terms of teaching experience, the 'junior' 
group did not have any, 'intermediates' had 1-4 years, 
and 'seniors' had 5 or more years. 

Teacher training 

Teachers who were assigned to the intervention group 
attended a two-day clinical teacher training, which was 
part of the faculty development programme of the 
Charite -Medical University of Berlin. Training was 
carried out by one to two experienced educators with a 
background in educational psychology (generic adult 
learning) or clinical teaching (clinical application). The 
learning contents were: 'role of the teacher', 'needs of 
learners', 'providing feedback', 'structure of session', 'defin- 
ing learning objectives', 'activating learners', 'teaching of 
skills', 'teaching with patients'. The courses contained 
10-14 participants, the training involved discussion 
groups, role-play, and reflection exercises (for details 
see Additional file 1: Table SI). The training was well 
received, attaining an overall rating by participants of 
85-90% of the possible maximum. 

Teaching sessions for the assessment of teaching per- 
formance after the training were scheduled two to three 
weeks later. 

Emergency medicine course 

The course of 12 full hours within two days followed a 
pre-defined structure. For a total of 16-18 students, 
each course involved a team of three teachers (junior, 
intermediate, senior). After an introductory interactive 
plenary session, students were divided into groups of 5-6. 
These groups rotated through 6 teaching sessions, each of 
which was taught by one member of the teacher team. 
Teaching formats were based on scenario teaching and 
discussion groups and focussed on the acquisition of prac- 
tical skills. Learning objectives (in accordance with the 
guidelines of the European Resuscitation Council) were: 
'initial assessment of the emergency patient! 'management 
of unconsciousness', 'basic life support (BLS)', 'use of 
automated external defibrillator (AED)', 'bag-mask- 
ventilation', and 'basic trauma management'. There was 
a formative assessment of BLS at the end of day 1 and a 
three-station standardized clinical examination (SCE) at 
the end of day 2. The SCE was mandatory to pass the 
course, but failure rates were known to be low from 
previous student cohorts. 

Characteristics of students 

We included only data from students who regularly 
completed the entire emergency medicine course and 
who were not enrolled in any external students' ex- 
change program. 
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Measures of evaluation 

Assessment of student skills and knowledge 

After the emergency medicine course student perform- 
ance was assessed by a three-station SCE (structured 
clinical examination) and an MCQ test. The SCE was 
the official assessment at the end of the course, while 
the MCQ was an additional test. 

From the SCE, we extracted four domains of skills, 
which were relevant within the test setting. This was done 
because scenario testing does not necessarily exhibit good 
content validity for all checklist items (e.g. 'check for 
safety' cannot easily be assessed under the safe conditions 
of an assessment situation). To enhance clinical relevance 
we only chose skills related to patient safety based on 
international resuscitation guidelines. Skill domains were: 
'alarm callj chest compressions! 'use of AED] 'bag-mask 
ventilation'. Each domain was analysed separately. For the 
total SCE score single domains were summed up. 

The MCQ included 14 single-choice questions, covering 
relevant content of the course, partly factual, partly pro- 
cedural knowledge. We limited the number of questions 
for reasons of time constraints and to ensure students' 
acceptance. The MCQ was validated by comparison with 
final year students within their anaesthesia clerkship (with 
97% of answers being correct). 

Evaluation of teachers by students 

The 30-item questionnaire was based on a review of the 
literature on teaching quality in medical education 
[5,17-22]. In accordance with two reviews in this field 
[17,20], we also linked our questions to two main fields 
of teaching performance: 'interpersonal communication' 
and 'structural aspects of teaching'. Ten questions dealt 
with student- teacher interaction, and 13 questions with 
teaching structure. Also, general acceptance of individual 
clinical teachers was rated (questionnaire see Additional 
file 2: Figure SI). Answers were given on a 7-point 
Likert-like scale (from +3 'strongly agree' to -3 'strongly 
disagree'). 

Teacher self-perception 

Teachers reported their perceptions of competence, self- 
confidence, and their overall satisfaction with their 
teaching session on a 7-point Likert-like scale. 

Informed consent 

All individual data of students and teachers were trans- 
formed into a pseudonymous format for all further steps 
of data processing to ensure that individuals could not 
be identified later. Teachers and students were formally 
asked to participate with the option to prohibit the use of 
their data at any time. All gave verbal consent. They were 
told that the study aimed to explore the effect of the com- 
position of the group of teachers. In order to minimize 



instrumentation bias they were not informed that the focus 
of the study was the teacher training. Approval was given 
by the educational board of Charite - Medical University of 
Berlin. 

Statistical analysis 

The sample size was calculated for a 15% difference of 
the total score of all four SCE domains. At a power level 
of 0.9 with an assumed alpha of 0.05, the sample size 
was set at 75 students per group. 

Results of SCE and MCQ were transformed into per- 
centages of the achievable maximum, and then medians, 
25 th and 75 th percentiles were calculated. Groups were 
compared using a Mann-Whitney-U-test. Significance 
was assumed at p-values below 0.05. The analyses were 
performed using SPSS 14.0. 

Results 

Characteristics of students 

From a total of 210 students, 17 were excluded from ana- 
lysis (13 students from external exchange programmes, 
4 with incomplete attendance). From the remaining 193 
students, 96 were taught by trained teachers, and 97 by 
untrained teachers. No statistical differences were found 
between the groups regarding age (median: 24 vs. 23 
years), female gender (56.3% vs. 59.6%), German as native 
language (85.4% vs. 86.6%), or previous experience in 
emergency medical services (13.7% vs. 11.3%), for details 
see Additional file 3: Table S2. 

Teacher characteristics 

All student courses were run exclusively with either 
trained teacher teams or untrained teams. Furthermore, 
all courses were taught by one junior, one intermediate, 
and one senior teacher. Due to clinical shifts, depart- 
ment rotations, and one teacher leaving the institution, 
the matched pair design could not be kept up entirely: 
three trained teachers did not teach within the context 
of the study, and one untrained teacher had to be re- 
cruited as an additional control. 

Trained teachers taught a median of 3 courses (range 
1-5), untrained teachers a median of 2 (1-4) courses. 
Time between training and first teaching encounter was 
14 days in median (range 6-41). Self-reported motiv- 
ation to teach was "high" to "very high" and did not dif- 
fer between the groups. Resulting teacher characteristics 
are shown in Table 1, the final study composition is 
shown in Additional file 4: Figure S2. 

Test results of students 

Practical skills: Standardized clinical examination (SCE) 

Four clinically relevant domains were extracted from SCE 
for analysis. The domains 'chest compressions' and 'use of 
AED' did not show significant differences (see Figure 1). 
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Table 1 Characteristics of individual clinical teachers 



Teachers 


Gender 


Time between Clinical experience 
training and (in years) 
first teaching 


Teaching experience 
(in years) 


Motivation 
to teach* 


Number of 
courses taught 


Junior pair 1 


f 


1 5 days 


1.5 


0 


+3 


1 




f 




] 


0 


+2.5 


3 


Junior pair 2 


m 


7 days 


< 1 


0 


+3 


5 




m 




< 1 


0 


+3 


2 


Junior pair 3 


f 


Training but no teaching 












f 




< 1 


0 


+3 


2 


Intermediate pair 1 


f 


6 days 


5 


3 


+2 


2 




f 




5 


3 


+3 


4 


Intermediate pair 2 


m 


27 days 


5 


3 


+3 


1 




m 




5 


3 


+2.5 


2 


Intermediate pair 3 


f 


Training but no teaching 












f 




4 


2 


n.a.** 


1 


Senior pair 1 


m 


13 days 


8 


6 


+3 


3 




m 




8 


6 


+3 


2 


Senior pair 2 


f 


41 days 


9 


6 


+2 


3 




f 




8 


6 


+3 


2 


Senior pair 3 


m 


Training but no teaching 












m 




/ 


5 


+3 


2 


Addit. control 


m 




10 


7 


+2 


1 



•Measured an a 7-point Likert scale ('how much do you appreciate teaching in general?' from -3: 'not at all' to +3: 'very strongly'). 
**Not answered. 



| scores of individual SCE domains | 

alarm chest AED bag-mask 

call compress use ventilation 

p<0.01 1 > p = 0.88 1 > p = 0.48 1 > p = 0.01 1 > 



n 



I I trained teachers I I untrained teachers 
Figure 1 Students' practical (SCE) and theoretical (MCQ) test results. 
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Notably in these domains both groups of students scored 
very high. 

For the domains 'alarm call' and 'bag-mask ventilation' 
students with untrained teachers scored significantly bet- 
ter than the trained group (p < 0.01 and p = 0.01). 

The overall SCE score was better for the untrained 
teacher group, but statistical significance was not reached 
(p = 0.06). Cronbach's Alpha for the sum of all four SCE 
domains was 0.77. 

Theoretical knowledge: MCQ 

In the MCQ, students of trained teachers gave correct 
answers in a median of 71.4% (57.1-83.9%, 25 th -75 th per- 
centile), while students from untrained teachers reached 
a median of 78.6% (64.3-85.7%). No statistical difference 
could be shown between the groups (p = 0.075), see 
Figure 1. For six single questions, results were statisti- 
cally different, for five of these in favor of the untrained 
teacher group. Questions representing procedural know- 
ledge showed no significant difference (Additional file 5: 
Table S3). Internal test consistency was calculated with a 
Cronbach's Alpha of 0.53. 

Evaluation of clinical teachers by students 

1) For the sum of all items for teaching quality, 
students of the trained teachers gave a significantly 
lower rating than the students of untrained teachers 
(68.0% (56.0-79.0%) vs. 72.0% (63.0-83.0%), p = 0.04, 
see Figure 2). 

2) For the sub-group of structural teaching aspects and 
for interpersonal communicative performance no 
statistical difference could be shown, but in both 



sub-domains a tendency towards lower ratings for 
the trained teachers was seen (teaching structure: 
76.7% (63.3-86.7%, 25 th -75 th percentile) vs. 83.3% 
(66.7-90.0%, p = 0.12; interpersonal communication: 
71.8% (59.0-84.6%) vs. 79.5% (66.7-87.2%) p = 0.08), 
see Figure 2). 

3) Results of the global rating of individual teachers are 
shown in Table 2. The group of trained 'senior' 
teachers again was rated significantly lower than the 
untrained seniors (2.57 vs. 2.74, p < 0.01). For 'juniors' 
and 'intermediates' no significant differences were 
shown. 

Self-perception of teachers 

Results are shown in Table 3 as qualitative data because 
every distinct category only included 6-7 ratings. Trained 
teachers appeared to feel more competent and more self- 
confident. This effect seems to be strongest for the 'inter- 
mediate' teachers. Overall satisfaction with the teaching 
session did not differ between the groups. 

Discussion 

In this study none of the measured parameters of teach- 
ing performance was found to be superior in the group 
of trained teachers. This finding specifically refers to the 
primary endpoint (practical skills of students), where 
two domains even revealed a significantly worse per- 
formance in the trained teacher group. Looking at the 
two other domains more closely (chest compressions 
and AED use), it has to be considered that students 
reached very high scores. This might indicate a ceiling 
effect, which can mask potential differences. On the 
other hand teachers might have strongly focussed on 



teaching structure 



interpersonal 
communication 



total evaluation 
score 



p = 0.12 



p = 0.08 



p = 0.04 



trained 



o ° 
o 

untrained teachers 



Figure 2 Evaluation of teaching quality by students. 
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Table 2 Evaluation of individual teachers by students 
(global performance rated on a 7-point Likert-like scale 



irom — j very 




+ 3 'very good') 






Global 




With training Without training 


p-value 


performance of 




(n = 96) 


(n = 97) 




Junior teacher 


Mean 


2.17 


2.28 


n.s. 




Median 


2 (2-3) 


3 (1-3) 


n.s. 


Intermediate 


Mean 


2.64 


2.71 


n.s. 


teacher 


Median 


3 (2-3) 


3 (2-3) 




n.s. 


Senior teacher 


Mean 


2.57 


2.74 1 ' 


p = 0.008 




Median 


3 (2-3) 


3 (3-3)" 


n.s. 



"n = 95. 



these core competencies in resuscitation, since they hold 
the highest evidence level in international resuscitation 
guidelines. 

The secondary endpoint 'theoretical knowledge' 
(MCQ) also showed no advantage of the trained group 
with a tendency towards inferior test results for the 
group of trained teachers. Evaluation of teaching quality 
by students revealed significantly lower ratings for 
trained teachers, with analogous trends for the sub- 
domains 'teaching structure' and 'interaction'. 

As stated at the beginning of this article, the most 
likely explanation for these negative results are difficul- 
ties in newly trained teachers putting acquired theoret- 
ical concepts into practice. After a two-day training with 
plenty of information and a generally strong shift in atti- 
tudes (from teacher- to learner-centeredness), teaching 
performance was assessed when the learners had not yet 
been given time to practice the newly-learned techniques. 



Table 3 Self perception of teachers 



Question 


Teacher's clinical 
competence 


Trained 


Untrained 


How competent 


Junior 


2.0 (6) 11 


1.7 (7) 


did you feel in 
your session? 


Intermediate 


2.5 (6) 


1.9 (7) 




Senior 


2.2 (6) 


2.0 (7) 




All competence levels 


2.2 (18) 


1.9 (21) 


How confident/sure 


Junior 


2.1 (6) 


1.8 (7) 


of yourself did you feel 
during your session? 


Intermediate 


2.5 (6) 


2.0 (7) 




Senior 


2.2 (6) 


2.3 (7) 




All competence levels 


2.3 (18) 


2.0 (21) 


How satisfied did you 


Junior 


1.5 (6) 


1.8 (7) 


feel with your session? 


Intermediate 


2.0 (6) 


1.7 (7) 




Senior 


1.9 (6) 


1.8 (7) 




All competence levels 


1.8 (18) 


1.8 (21) 



Mean ratings on a 7-point Likert-like scale from -3: 'not at all' to + 3 'very 
much/strong'; in brackets: number of evlauations. 



Thus a 'confusion effect' may have occurred, making it 
impossible to adequately integrate the contents of the 
training, resulting in poorer teaching performance. This 
hypothesis may be supported by the self-perception of 
trained teachers, who did not feel more satisfied with their 
sessions although they reported higher levels of compe- 
tence and self-confidence. A similar phenomenon was de- 
scribed in the field of business school education, where 
business school students performed inferior to untrained 
controls, when they had to transfer their substantial know- 
ledge to practical applications [13,23]. 

One might speculate that teaching quality would have 
improved if it had been assessed at a later time. Unfortu- 
nately re-evaluation of teachers was not possible because 
some of the teachers had since left the institution. 

Our findings indicate that duration and timing - the 
"dosage" - of teacher training need to be considered 
carefully. Other training programmes of longer duration 
have been proven to be efficient [5,24], and even some 
shorter programmes have improved teaching perform- 
ance [6,9,25]. However, these programmes were predom- 
inantly designed for resident and clerkship teaching, and 
teaching quality was assessed at a later time point long 
after training, offering considerable practical opportunity 
to integrate new competencies. 

Another aspect might be the distribution of training 
over time. While many courses are run as one or two- 
day seminars or workshops, Morrison et al. spread their 
training of only 13 hours over 6 months and substan- 
tially improved the score in an objective standardized 
teaching examination [3]. In this light, shorter training 
units might be more "digestible", hence in our two-day 
training we might have administered an "overdose". 
However, whether teaching performance might have 
improved over time remains an open question. 

Potential limitations 
Measurement validity 

For the primary endpoint, the students' practical test re- 
sults, good validity can be assumed, since the applied 
SCE domains had good content validity and good in- 
ternal consistency. Furthermore, the calculated sample 
size was reached. 

Regarding MCQ results, internal consistency of the scale 
was low, most likely due to the small number of items. This 
restriction followed practical reasoning in order to keep the 
students' workload tolerable. We also acknowledge that 
while observing a group effect, the low Cronbach's Alpha 
value allows only preliminary interpretations. 

As a further limitation, the questionnaire for teacher 
evaluations was not validated. Nevertheless it was sup- 
ported by sound literature and was specifically tailored to 
the study setting. Sub-analysis was also supported by the 
literature [17,20]. The gender distribution of teachers was 
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imbalanced, with a larger number of female teachers. 
Research, however, does not suggest a substantial effect of 
gender on teaching quality [26,27]. Factors such as teach- 
ing experience probably play a greater role. 

Appropriateness of the teacher training for the teaching 
format 

Although there is little standardization of teacher train- 
ing, we assume for our training that learning contents 
and qualifications of educators met common standards. 
Learning contents covered at least 75% of the teaching 
situations, which explicitly occurred in the studied emer- 
gency medicine course (Additional file 1: Table SI). Per- 
haps the training may have shifted the teachers' focus 
from pure skill training to a more holistic teaching ap- 
proach fostering deep learning. In this case, the practical 
skills of students might not represent a relevant outcome 
for contents delivered by the training. However, evaluation 
of teaching quality by students (including student-teacher 
interaction) did not indicate a more learner-centred atti- 
tude of trained teachers. The impact of the intervention is 
additionally supported by the trained teachers reporting a 
higher degree of self-confidence. 

The pre-structured format of the emergency medicine 
course might itself have had an influence on the teach- 
ing quality providing a curricular setting for effective 
learning. Notably, all clinical teachers achieved high rat- 
ings. Further reasons for favourable ratings could have 
been high initial teaching competencies [28] and the fact 
that students strongly value the practical teaching in 
emergency medicine [29]. However, these points would 
not explain the inferior teaching performance of trained 
teachers. Nonetheless, the curricular structure might 
have hampered the individual creativity of teachers by 
defining learning objectives, materials and methods in 
advance. 

Generalizability 

Primarily the study results are restricted to a single insti- 
tution since the teacher training may be not comparable 
to others. Nonetheless, we think that the main finding of 
this study - the lack of improvement in the trained 
teacher group - is consistent over the different study 
outcomes. This finding appears important, since the 
mechanisms of acquisition of competencies from teacher 
training sessions are not well understood. It is possible 
that specific circumstances of the study design brought 
to sight an early stadium of transition during learning of 
complex skills. The results of this study should advise us 
to think more thoroughly about timing, duration, and 
amount of content of teacher trainings. At least, it could 
be of value to know about the possibility that teaching 
performance does not necessarily improve immediately 
after training. Further investigations should explore this 



field in more depth and focus on the longitudinal ana- 
lysis of teacher development. 

Conclusions 

In this prospective study, outcomes after clinical teacher 
training teaching were not found to be superior com- 
pared to untrained controls. For a number of study end 
points, results even turned out to be significantly worse. 
A potential reason could be that trained teachers experi- 
enced difficulties in integrating new teaching competen- 
cies into their teaching practice at this time point. 
Further research is warranted to clarify the underlying 
mechanisms. 

Additional files 



Additional file 1: Table SI. Structure of teacher training (overview). 
Additional file 2: Figure SI. Questionnaire for evaluation by students. 
Additional file 3: Table S2. Characteristics of students. 
Additional file 4: Figure S2. Resulting final study configuration. 
Additional file 5: Table S3. MCQ by single guestions. 
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